Finding Statistically Significant Attribute Interactions

In many data exploration tasks it is meaningful to identify groups of aŠribute interactions that are speci€c to a variable of interest. For instance, in a dataset where the aŠributes are medical markers and the variable of interest (class variable) is binary indicating presence/absence of disease, we would like to know which medical markers interact with respect to the binary class label. Œese interactions are useful in several practical applications, for example, to gain insight into the structure of the data, in feature selection, and in data anonymisation. We present a novel method, based on statistical signi€cance testing, that can be used to verify if the data set has been created by a given factorised class-conditional joint distribution, where the distribution is parametrised by a partition of its aŠributes. Furthermore, we provide a method, named astrid, for automatically €nding a partition of aŠributes describing the distribution that has generated the data. State-of-the-art classi€ers are utilised to capture the interactions present in the data by systematically breaking aŠribute interactions and observing the e‚ect of this breaking on classi€er performance. We empirically demonstrate the utility of the proposed method with examples using real and synthetic data.